Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP new testing infrastructure #3242

Closed
wants to merge 36 commits into from
Closed

WIP new testing infrastructure #3242

wants to merge 36 commits into from

Conversation

KyleAMathews
Copy link
Contributor

We've been using (abusing?) Netlify for testing building example sites as integration tests.

This has a few problems:

  • Netlify logs aren't easily accessible to people other than me
  • Netlify limits how fast we can build sites
  • It's not obvious if a site fails because Netlify adding a comment on every site build got super noisy so I had to turn it off. So sites can stop building and we won't notice it for a while.
  • There's a lot of manual work to setup each new site — would much prefer it's automatic — just build every site in the examples directory
  • We can't extend what we're testing e.g. would be nice to calculate hash of every generated file and warn if things change.
  • Also really want to add build & client performance testing — track build length, max-memory used, various frontend performance metrics on e.g. webpagetest
  • Also would like to add automated client testing w/ Chrome Puppeteer. Much easier to do this if we control the infrastructure

@ghost ghost assigned KyleAMathews Dec 16, 2017
@ghost ghost added the review label Dec 16, 2017
@KyleAMathews
Copy link
Contributor Author

This is part 2 of a PR last month #2920

@gatsbybot
Copy link
Collaborator

gatsbybot commented Dec 16, 2017

Deploy preview ready!

Built with commit 5e2482c

https://deploy-preview-3242--gatsbygram.netlify.com

@KyleAMathews
Copy link
Contributor Author

Oh didn't finish my explanation :-)

So what this PR will add is we'll build every commit of every PR on
AWS CodeBuild and we'll report back the result in a comment (and updating it as more commits are added).

This will make it easy to see at a glance if all example sites are building giving us more confidence that PRs are solid.

On top of this, we'll add performance testing both for builds & frontend loading.

@jquense
Copy link
Contributor

jquense commented Dec 22, 2017

🔥

@KyleAMathews
Copy link
Contributor Author

There's some good stuff here and we're definitely going to do this but we'll be doing it in a somewhat different way so closing this for now.

@ghost ghost removed the review label Apr 4, 2018
@m-allanson m-allanson deleted the infrastructure branch May 9, 2018 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants